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Claim Amendments: 

1 . (Currently Amended) A method for evaluating group-administered test results, said 

method comprising: 

(a) providing a set of class profiles by test administrator, each class profile 
indicative of the question bv question performance of a respective class on a selected group- 
administered test comprising a plurality of questions which are di rected to measure the same 
characteristic; and 

(b) generating with a computer a normative profile indicative of normative 
class performance on the selected test, where the members of each class included in the 
normative profile are subject to the direction of a the same test administrator on the selected test; 

(c) comparing each of the class profiles with the normative profile on a 
question-by-question basis : and 

(d) generating a measure of the question bv q uestion variation of each of the 

class profiles from the normative profile . 

2. (Currently Amended) A method for evaluating group-administered test results, said 
method comprising: 

(a) providing a set of class profiles by test administrator, each class profile 
indicative of the question bv question p erformance of a respective class on a selected group- 
administered test comprising a plurality of question s which are directed to measure the same 
characteristic : and 

(b) generating with a computer a normative profile indicative of normative 
class performance on the selected test, where the members of each class included in the 
normative profile are subject to the direction of a the same test administrator on the selected test; 

(c) generating a measure of the question bv question variat ion of each of the 
class profiles from the normative profile; and 

(6) identifying individual ones of the class profiles that deviate excessively 
from the normative profile on a question-by-question basis. 

3. (Original) The method of Claim 1 or 2 wherein act (c) comprises identiiying selected 
ones of the class profiles that are characterized by a low correlation with the normative profile. 
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4. (Original) The method of Claim 1 or 2 wherein act (c) comprises comparing (1) at 
least two portions of each class profile that differ in question difficulty against (2) corresponding 
portions of the normative profile. 

5. (Original) The method of Claim 1 or 2 wherein act (c) comprises comparing (1) a 
difference between at least a portion of each of the class profiles and a corresponding portion of 
the normative profile on a question-by-question basis; and (2) a measure of expected variability 
of class performance for the corresponding portion of the class profiles. 

6. (Original) The method of Claim I or 2 wherein act (c) comprises comparing (1) a 
measure of difference between a first portion of each of the class profiles and a corresponding 
first portion of the normative profile and (2) a measure of difference between a second portion of 
each of the class profiles and a corresponding second portion of the normative profile. 

7. (Previously Amended) The method of Claim 1 or 2 wherein act (c) comprises 
comparing the frequency of successive equal measures of class success in answering the 
questions on a question-by-question basis. 

8. (Original) The method of Claim 1 or 2 wherein act (a) comprises: 

(al) providing a respective individual profile for each of a plurality of 
individuals that took the group-administered test, each individual associated with a respective 
one of the classes, each individual profile indicative of accuracy of the respective individual in 
answering the plurality of questions on a question-by-question basis; and 

(a2) determining each class profile from the individual profiles of the 
individuals in the respective class, each class profile indicative of accuracy of the individuals in 
the respective class in answering the questions on a question-by-question basis. 

9. (Original) The method of Claim 1 or 2 wherein the class profiles of (a) are each 
characterized by a skill level, and wherein the normative profile of (b) is indicative of expected 
class performance for a class of said skill level. 

1 0* (Original) The method of Claim 1 or 2 wherein the normative profile of (b) is 
indicative of expected class performance for classes of a wide range of skill levels. 
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1 1 . (Currently Amended) A method for evaluating standardized test results, said method 
comprising: 

(a) providing a set of sub-group profiles, each sub-group comprising a 
plurality of individuals whose test item question responses are exposed to a common element 
affecting test administration and each sub-group profile indicative of the question by question 
performance of a respective sub-group on a selected standardized test comprising a plurality of 
question s which are directed to measure the same characteristic: 

(b) generating with a computer a normative profile indicative of normative 
sub-group performance on the selected test; and 

(c) evaluating administration of the selected test based on a comparison of 
each of the sub-group profiles from the normative profile on a question-by-question basis. 

12. (Original) The method of Claim 1 1 wherein each sub-group comprises individuals to 
whom the selected test was administered by a respective test administrator in a respective testing 
session. 

13. (Original) The method of Claim 1 1 or 12 wherein act (c) comprises identifying 
selected ones of the sub-group profiles that are characterized by a low correlation with the 
normative profile. 

14. (Original) The method of Claim 1 1 or 12 wherein act (c) comprises comparing (1) at 
least two portions of each sub-group profile that differ in question difficulty against 

(2) corresponding portions of the normative profile. 

15. (Original) The method of Claim 11 or 12 wherein act (c) comprises comparing (1) a 
difference between at least a portion of each of the sub-group profiles and a corresponding 
portion of the normative profile on a question-by-question basis; and (2) a measure of expected 
variability of sub-group performance for the corresponding portion of the sub-group profiles. 

16. (Original) The method of Claim 11 or 12 wherein act (c) comprises comparing (1) a 
measure of difference between a first portion of each of the sub-group profiles and a 
corresponding first portion of the normative profile and (2) a measure of difference between a 
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second portion of each of the sub-group profiles and a corresponding second portion of the 
normative profile, 

17. (Previously Amended) The method of Claim 1 1 or 12 wherein act (c) comprises 
comparing the frequency of successive equal measures of class success in answering the 
questions on a question-by-question basis. 

18. (Original) The method of Claim 11 or 12 wherein act (a) comprises: 

(al) providing a respective individual profile for each of a plurality of 
individuals that took the standardized test each individual associated with a respective one of the 
sub-groups, each individual profile indicative of accuracy of the respective individual in 
answering the plurality of questions on a question-by-question basis; and 

(a2) determining each sub-group profile from the individual profiles of the 
individuals in the respective sub-group, each sub-group profile indicative of accuracy of the 
individuals in the respective sub-group in answering the questions on a question-by-question 
basis, 

19. (Original) The method of Claim 1 1 or 12 wherein the sub-group profiles of (a) are 
each characterized by a skill level, and wherein the normative profile of (b) is indicative of 
expected sub-group performance for a sub-group of said skill level. 

20. (Original) The method of Claim 1 1 or 12 wherein the normative profile of (b) is 
indicative of expected sub-group performance for sub-groups of a wide range of skill levels. 
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